An operator of a powered industrial vehicle is required to notice pedestrians in the area around which the vehicle is operating. Prior art solutions for detecting pedestrians include the use of electromagnetic radiation emitters coupled with RADAR sensors, laser sensors, and/or SONAR (ultrasonic) sensors to provide the operator with some indication that a pedestrian may be present in the area around the vehicle.
A significant limitation of prior art systems is in the inability to discern whether the object being detected is an insignificant inanimate object (e.g. trash, boxes, poles, etc.) or a pedestrian. As a result, prior art systems alert the industrial truck operator of the presence of every object thereby creating multiple false alarms. These false alarms annoy the operator with unnecessary warnings, and cause the operator to be less sensitive to the warnings.
Furthermore, radar and laser sensors as well as ultrasound sensors have the disadvantage that in the immediate vehicle surroundings they are able to detect only a small region of the surroundings because of their small aperture angle, which typically provides a narrow FOV. Thus, a large number of sensors is required if the entire vehicle surroundings are to be detected using such sensors.
An example of a laser based system, is disclosed in U.S. Pat. No. 7,164,118 (hereinafter U.S. '118), by Anderson et al U.S. '118 discloses a method of detecting presence of an object and the distance between the system and an object using a laser mounted on an industrial vehicle. The transmitter emits linear beams of electromagnetic radiation with a transmitted radiation pattern within a defined spatial zone. A camera collects an image of the defined spatial zone. A data processor detects a presence of an object in the collected image based on an observed illumination radiation pattern on an object formed by at least one of the linear beams. A distance estimator estimates a distance between the object and the optical device.
There are also prior art systems using imaging devices to image the scene in an angle 360° horizontally around a vehicle. Such a system is disclosed in U.S. patent application 2004/0075544 (hereinafter U.S. '544), by Janssen Holger. U.S. '544 uses two optical sensors that act as a pair of stereo cameras. The sensors are coupled with fisheye lenses, which have a very wide-angle of 220°. Thus, a large portion of the surroundings of the motor vehicle may be detected but the very wide-angle lenses provide images with a large extend of distortion, and U.S. '544 does not disclose if the distortion is corrected. In U.S. '544 all sensors emit the sensed information to a single controller.
Tracking a detected pedestrian over time enables the system to detect a pedestrian at a relatively far distance from the vehicle, such as 15 meters or more, and then track the detected pedestrian with high confidence at a closer range, which might endanger the pedestrian and thus, the powered industrial vehicle driver will be warned by the system. Tracking also enables the system to stay locked on a detected pedestrian as the image of a detected pedestrian departs from a frame provided by one camera and enters a frame of an adjacent camera of the same system. Tracking of the detected pedestrian will then proceed using the second camera.
There are prior art systems, mounted in vehicles, for detecting pedestrians and for measuring the distance from the vehicle to the detected pedestrian. A pedestrian detection system is described in U.S. application Ser. No. 10/599,635 (hereinafter U.S. '635) by Shashua et al, the disclosure of which is included herein by reference for all purposes as if entirely set forth herein. U.S. '635 provides a system mounted on a host vehicle and methods for detecting pedestrians in an image frame, the image provided by a monocular camera.
A distance measurement from a visible camera image frame is described in “Vision based ACC with a Single Camera: Bounds on Range and Range Rate Accuracy” by Stein et al., presented at the IEEE Intelligent Vehicles Symposium (IV2003), the disclosure of which is incorporated herein by reference for all purposes as if entirely set forth herein. Distance measurement is further discussed in U.S. application Ser. No. 11/554,048 (hereinafter U.S. '048) by Stein et al., the disclosure of which is included herein by reference for all purposes as if entirely set forth herein. U.S. '048 provides methods for refining distance measurements from the vehicle hosting the distance measuring system, to an obstruction.
An obstruction detection and tracking system is described in U.S. Pat. No. 7,113,867 (hereinafter U.S. '867) by Stein, and included herein by reference for all purposes as if entirely set forth herein. Obstruction detection and tracking is performed based on information from multiple images captured in real time using a camera mounted in a vehicle hosting the obstruction detection and tracking system.
The systems disclosed in U.S. '635, U.S. '867 and U.S. '048, are typically part of a warning and/or control system for vehicles that are typically traveling forward on roads at relatively high speeds. They are not suitable to a powered industrial vehicle, such as a forklift, which typically travels off the road, at low speeds and in any directions. Thus, a powered industrial vehicle needs a warning system that can warn the driver of a pedestrian located anywhere near in the area around the powered industrial vehicle.
Thus, there is a need for and it would be advantageous to have a system including multiple cameras mounted on a powered industrial truck, each camera equipped with an image processing system for detecting pedestrians and in the system when one camera detects a pedestrian and the pedestrian moves out of the field of view (in horizontal plane) of the one camera, data is passed to the second camera so that the pedestrian is tracked using the multiple cameras over a wide field of view.
The term “powered industrial vehicle” as used herein refers to a vehicle selected from the group of vehicles including forklifts, container handlers, rubber tired gantry cranes. A powered industrial vehicle typically travels at a low speed, is capable of moving in multiple directions and frequently changes the traveling direction.
The term “Field Of View” (FOV) in general is the angular extent of a given scene, delineated by the angle of a three dimensional cone that is imaged onto an image sensor of a camera, the camera being the vertex of the three dimensional cone. The FOV of a camera at particular distances is determined by the focal length of the lens: the longer the focal length, the narrower the field of view. The terms “Field Of View” of a camera and “viewing zone” of a camera are used herein interchangeably and are used herein to refer to the horizontal angular extent of a given scene, as imaged on to the image sensor of the camera. It is assumed that the dimensions of the detector are adapted to the camera FOV.